The Thoracic Research Evaluation and Treatment 2.0 Model: A Lung Cancer Prediction Model for Indeterminate Nodules Referred for Specialist Evaluation.

BACKGROUND: Appropriate risk stratification of indeterminate pulmonary nodules (IPNs) is necessary to direct diagnostic evaluation. Currently available models were developed in populations with lower cancer prevalence than that seen in thoracic surgery and pulmonology clinics and usually do not allow for missing data. We updated and expanded the Thoracic Research Evaluation and Treatment (TREAT) model into a more generalized, robust approach for lung cancer prediction in patients referred for specialty evaluation. RESEARCH QUESTION: Can clinic-level differences in nodule evaluation be incorporated to improve lung cancer prediction accuracy in patients seeking immediate specialty evaluation compared with currently available models? STUDY DESIGN AND METHODS: Clinical and radiographic data on patients with IPNs from six sites (N = 1,401) were collected retrospectively and divided into groups by clinical setting: pulmonary nodule clinic (n = 374; cancer prevalence, 42%), outpatient thoracic surgery clinic (n = 553; cancer prevalence, 73%), or inpatient surgical resection (n = 474; cancer prevalence, 90%). A new prediction model was developed using a missing data-driven pattern submodel approach. Discrimination and calibration were estimated with cross-validation and were compared with the original TREAT, Mayo Clinic, Herder, and Brock models. Reclassification was assessed with bias-corrected clinical net reclassification index and reclassification plots. RESULTS: Two-thirds of patients had missing data; nodule growth and fluorodeoxyglucose-PET scan avidity were missing most frequently. The TREAT version 2.0 mean area under the receiver operating characteristic curve across missingness patterns was 0.85 compared with that of the original TREAT (0.80), Herder (0.73), Mayo Clinic (0.72), and Brock (0.68) models with improved calibration. The bias-corrected clinical net reclassification index was 0.23. INTERPRETATION: The TREAT 2.0 model is more accurate and better calibrated for predicting lung cancer in high-risk IPNs than the Mayo, Herder, or Brock models. Nodule calculators such as TREAT 2.0 that account for varied lung cancer prevalence and that consider missing data may provide more accurate risk stratification for patients seeking evaluation at specialty nodule evaluation clinics.

Covariate-specific ROC curve analysis can accommodate differences between covariate subgroups in the evaluation of diagnostic accuracy.

OBJECTIVES: We present an illustrative application of methods that account for covariates in receiver operating characteristic (ROC) curve analysis, using individual patient data on D-dimer testing for excluding pulmonary embolism. STUDY DESIGN AND SETTING: Bayesian nonparametric covariate-specific ROC curves were constructed to examine the performance/positivity thresholds in covariate subgroups. Standard ROC curves were constructed. Three scenarios were outlined based on comparison between subgroups and standard ROC curve conclusion: (1) identical distribution/identical performance, (2) different distribution/identical performance, and (3) different distribution/different performance. Scenarios were illustrated using clinical covariates. Covariate-adjusted ROC curves were also constructed. RESULTS: Age groups had prominent differences in D-dimer concentration, paired with differences in performance (Scenario 3). Different positivity thresholds were required to achieve the same level of sensitivity. D-dimer had identical performance, but different distributions for YEARS algorithm items (Scenario 2), and similar distributions for sex (Scenario 1). For the later covariates, comparable positivity thresholds achieved the same sensitivity. All covariate-adjusted models had AUCs comparable to the standard approach. CONCLUSION: Subgroup differences in performance and distribution of results can indicate that the conventional ROC curve is not a fair representation of test performance. Estimating conditional ROC curves can improve the ability to select thresholds with greater applicability.

Racial Disparities in Lung Cancer Stage of Diagnosis Among Adults Living in the Southeastern United States.

BACKGROUND: Black Americans receive a diagnosis at later stage of lung cancer more often than White Americans. We undertook a population-based study to identify factors contributing to racial disparities in lung cancer stage of diagnosis among low-income adults. RESEARCH QUESTION: Which multilevel factors contribute to racial disparities in stage of lung cancer at diagnosis? STUDY DESIGN AND METHODS: Cases of incident lung cancer from the prospective observational Southern Community Cohort Study were identified by linkage with state cancer registries in 12 southeastern states. Logistic regression shrinkage techniques were implemented to identify individual-level and area-level factors associated with distant stage diagnosis. A subset of participants who responded to psychosocial questions (eg, racial discrimination experiences) were evaluated to determine if model predictive power improved. RESULTS: We identified 1,572 patients with incident lung cancer with available lung cancer stage (64% self-identified as Black and 36% self-identified as White). Overall, Black participants with lung cancer showed greater unadjusted odds of distant stage diagnosis compared with White participants (OR,1.29; 95% CI, 1.05-1.59). Greater neighborhood area deprivation was associated with distant stage diagnosis (OR, 1.58; 95% CI, 1.19-2.11). After controlling for individual- and area-level factors, no significant difference were found in distant stage disease for Black vs White participants. However, participants with COPD showed lower odds of distant stage diagnosis in the primary model (OR, 0.72; 95% CI, 0.53-0.98). Interesting and complex interactions were observed. The subset analysis model with additional variables for racial discrimination experiences showed slightly greater predictive power than the primary model. INTERPRETATION: Reducing racial disparities in lung cancer stage at presentation will require interventions on both structural and individual-level factors.

Evaluation of resampling-based inference for topological features of neuroimages.

Many recent studies have demonstrated the inflated type 1 error rate of the original Gaussian random field (GRF) methods for inference of neuroimages and identified resampling (permutation and bootstrapping) methods that have better performance. There has been no evaluation of resampling procedures when using robust (sandwich) statistical images with different topological features (TF) used for neuroimaging inference. Here, we consider estimation of distributions TFs of a statistical image and evaluate resampling procedures that can be used when exchangeability is violated. We compare the methods using realistic simulations and study sex differences in life-span age-related changes in gray matter volume in the Nathan Kline Institute Rockland sample. We find that our proposed wild bootstrap and the commonly used permutation procedure perform well in sample sizes above 50 under realistic simulations with heteroskedasticity. The Rademacher wild bootstrap has fewer assumptions than the permutation and performs similarly in samples of 100 or more, so is valid in a broader range of conditions. We also evaluate the GRF-based pTFCE method and show that it has inflated error rates in samples less than 200. Our R package, pbj , is available on Github and allows the user to reproducibly implement various resampling-based group level neuroimage analyses.

FDRestimation: Flexible False Discovery Rate Computation in R.

False discovery rates (FDR) are an essential component of statistical inference, representing the propensity for an observed result to be mistaken. FDR estimates should accompany observed results to help the user contextualize the relevance and potential impact of findings. This paper introduces a new user-friendly R pack-age for estimating FDRs and computing adjusted p-values for FDR control. The roles of these two quantities are often confused in practice and some software packages even report the adjusted p-values as the estimated FDRs. A key contribution of this package is that it distinguishes between these two quantities while also offering a broad array of refined algorithms for estimating them. For example, included are newly augmented methods for estimating the null proportion of findings - an important part of the FDR estimation procedure. The package is broad, encompassing a variety of adjustment methods for FDR estimation and FDR control, and includes plotting functions for easy display of results. Through extensive illustrations, we strongly encourage wider reporting of false discovery rates for observed findings.

Estimating the Posttest Probability of Long QT Syndrome Diagnosis for Rare KCNH2 Variants.

BACKGROUND: The proliferation of genetic profiling has revealed many associations between genetic variations and disease. However, large-scale phenotyping efforts in largely healthy populations, coupled with DNA sequencing, suggest variants currently annotated as pathogenic are more common in healthy populations than previously thought. In addition, novel and rare variants are frequently observed in genes associated with disease both in healthy individuals and those under suspicion of disease. This raises the question of whether these variants can be useful predictors of disease. To answer this question, we assessed the degree to which the presence of a variant in the cardiac potassium channel gene KCNH2 was diagnostically predictive for the autosomal dominant long QT syndrome. METHODS: We estimated the probability of a long QT diagnosis given the presence of each KCNH2 variant using Bayesian methods that incorporated variant features such as changes in variant function, protein structure, and in silico predictions. We call this estimate the posttest probability of disease. Our method was applied to over 4000 individuals heterozygous for 871 missense or in-frame insertion/deletion variants in KCNH2 and validated against a separate international cohort of 933 individuals heterozygous for 266 missense or in-frame insertion/deletion variants. RESULTS: Our method was well-calibrated for the observed fraction of heterozygotes diagnosed with long QT syndrome. Heuristically, we found that the innate diagnostic information one learns about a variant from 3-dimensional variant location, in vitro functional data, and in silico predictors is equivalent to the diagnostic information one learns about that same variant by clinically phenotyping 10 heterozygotes. Most importantly, these data can be obtained in the absence of any clinical observations. CONCLUSIONS: We show how variant-specific features can inform a prior probability of disease for rare variants even in the absence of clinically phenotyped heterozygotes.

Effect of balanced crystalloids versus saline on urinary biomarkers of acute kidney injury in critically ill adults.

BACKGROUND: Recent trials have suggested use of balanced crystalloids may decrease the incidence of major adverse kidney events compared to saline in critically ill adults. The effect of crystalloid composition on biomarkers of early acute kidney injury remains unknown. METHODS: From February 15 to July 15, 2016, we conducted an ancillary study to the Isotonic Solutions and Major Adverse Renal Events Trial (SMART) comparing the effect of balanced crystalloids versus saline on urinary levels of neutrophil gelatinase-associated lipocalin (NGAL) and kidney injury molecule-1 (KIM-1) among 261 consecutively-enrolled critically ill adults admitted from the emergency department to the medical ICU. After informed consent, we collected urine 36 ± 12 h after hospital admission and measured NGAL and KIM-1 levels using commercially available ELISAs. Levels of NGAL and KIM-1 at 36 ± 12 h were compared between patients assigned to balanced crystalloids versus saline using a Mann-Whitney U test. RESULTS: The 131 patients (50.2%) assigned to the balanced crystalloid group and the 130 patients (49.8%) assigned to the saline group were similar at baseline. Urinary NGAL levels were significantly lower in the balanced crystalloid group (median, 39.4 ng/mg [IQR 9.9 to 133.2]) compared with the saline group (median, 64.4 ng/mg [IQR 27.6 to 339.9]) (P < 0.001). Urinary KIM-1 levels did not significantly differ between the balanced crystalloid group (median, 2.7 ng/mg [IQR 1.5 to 4.9]) and the saline group (median, 2.4 ng/mg [IQR 1.3 to 5.0]) (P = 0.36). CONCLUSIONS: In this ancillary analysis of a clinical trial comparing balanced crystalloids to saline among critically ill adults, balanced crystalloids were associated with lower urinary concentrations of NGAL and similar urinary concentrations of KIM-1, compared with saline. These results suggest only a modest reduction in early biomarkers of acute kidney injury with use of balanced crystalloids compared with saline. TRIAL REGISTRATION: ClinicalTrials.gov number: NCT02444988 . Date registered: May 15, 2015.

A Bayesian method to estimate variant-induced disease penetrance.

A major challenge emerging in genomic medicine is how to assess best disease risk from rare or novel variants found in disease-related genes. The expanding volume of data generated by very large phenotyping efforts coupled to DNA sequence data presents an opportunity to reinterpret genetic liability of disease risk. Here we propose a framework to estimate the probability of disease given the presence of a genetic variant conditioned on features of that variant. We refer to this as the penetrance, the fraction of all variant heterozygotes that will present with disease. We demonstrate this methodology using a well-established disease-gene pair, the cardiac sodium channel gene SCN5A and the heart arrhythmia Brugada syndrome. From a review of 756 publications, we developed a pattern mixture algorithm, based on a Bayesian Beta-Binomial model, to generate SCN5A penetrance probabilities for the Brugada syndrome conditioned on variant-specific attributes. These probabilities are determined from variant-specific features (e.g. function, structural context, and sequence conservation) and from observations of affected and unaffected heterozygotes. Variant functional perturbation and structural context prove most predictive of Brugada syndrome penetrance.

A Robust Effect Size Index.

Effect size indices are useful tools in study design and reporting because they are unitless measures of association strength that do not depend on sample size. Existing effect size indices are developed for particular parametric models or population parameters. Here, we propose a robust effect size index based on M-estimators. This approach yields an index that is very generalizable because it is unitless across a wide range of models. We demonstrate that the new index is a function of Cohen's d, [Formula: see text], and standardized log odds ratio when each of the parametric models is correctly specified. We show that existing effect size estimators are biased when the parametric models are incorrect (e.g., under unknown heteroskedasticity). We provide simple formulas to compute power and sample size and use simulations to assess the bias and standard error of the effect size estimator in finite samples. Because the new index is invariant across models, it has the potential to make communication and comprehension of effect size uniform across the behavioral sciences.

Semi-supervised Machine Learning with MixMatch and Equivalence Classes.

Semi-supervised methods have an increasing impact on computer vision tasks to make use of scarce labels on large datasets, yet these approaches have not been well translated to medical imaging. Of particular interest, the MixMatch method achieves significant performance improvement over popular semi-supervised learning methods with scarce labels in the CIFAR-10 dataset. In a complementary approach, Nullspace Tuning on equivalence classes offers the potential to leverage multiple subject scans when the ground truth for the subject is unknown. This work is the first to (1) explore MixMatch with Nullspace Tuning in the context of medical imaging and (2) characterize the impacts of the methods with diminishing labels. We consider two distinct medical imaging domains: skin lesion diagnosis and lung cancer prediction. In both cases we evaluate models trained with diminishing labeled data using supervised, MixMatch, and Nullspace Tuning methods as well as MixMatch with Nullspace Tuning together. MixMatch with Nullspace Tuning together is able to achieve an AUC of 0.755 in lung cancer diagnosis with only 200 labeled subjects on the National Lung Screening Trial and a balanced multi-class accuracy of 77% with only 779 labeled examples on HAM10000. This performance is similar to that of the fully supervised methods when all labels are available. In advancing data driven methods in medical imaging, it is important to consider the use of current state-of-the-art semi-supervised learning methods from the greater machine learning community and their impact on the limitations of data acquisition and annotation.

Missing data and prediction: the pattern submodel.

Missing data are a common problem for both the construction and implementation of a prediction algorithm. Pattern submodels (PS)-a set of submodels for every missing data pattern that are fit using only data from that pattern-are a computationally efficient remedy for handling missing data at both stages. Here, we show that PS (i) retain their predictive accuracy even when the missing data mechanism is not missing at random (MAR) and (ii) yield an algorithm that is the most predictive among all standard missing data strategies. Specifically, we show that the expected loss of a forecasting algorithm is minimized when each pattern-specific loss is minimized. Simulations and a re-analysis of the SUPPORT study confirms that PS generally outperforms zero-imputation, mean-imputation, complete-case analysis, complete-case submodels, and even multiple imputation (MI). The degree of improvement is highly dependent on the missingness mechanism and the effect size of missing predictors. When the data are MAR, MI can yield comparable forecasting performance but generally requires a larger computational cost. We also show that predictions from the PS approach are equivalent to the limiting predictions for a MI procedure that is dependent on missingness indicators (the MIMI model). The focus of this article is on out-of-sample prediction; implications for model inference are only briefly explored.

Discovering novel disease comorbidities using electronic medical records.

Increasing reliance on electronic medical records at large medical centers provides unique opportunities to perform population level analyses exploring disease progression and etiology. The massive accumulation of diagnostic, procedure, and laboratory codes in one place has enabled the exploration of co-occurring conditions, their risk factors, and potential prognostic factors. While most of the readily identifiable associations in medical records are (now) well known to the scientific community, there is no doubt many more relationships are still to be uncovered in EMR data. In this paper, we introduce a novel finding index to help with that task. This new index uses data mined from real-time PubMed abstracts to indicate the extent to which empirically discovered associations are already known (i.e., present in the scientific literature). Our methods leverage second-generation p-values, which better identify associations that are truly clinically meaningful. We illustrate our new method with three examples: Autism Spectrum Disorder, Alzheimer's Disease, and Optic Neuritis. Our results demonstrate wide utility for identifying new associations in EMR data that have the highest priority among the complex web of correlations and causalities. Data scientists and clinicians can work together more effectively to discover novel associations that are both empirically reliable and clinically understudied.

Evaluation of USPSTF Lung Cancer Screening Guidelines Among African American Adult Smokers.

IMPORTANCE: The United States Preventive Services Task Force (USPSTF) recommends low-dose computed tomography screening for lung cancer. However, USPSTF screening guidelines were derived from a study population including only 4% African American smokers, and racial differences in smoking patterns were not considered. OBJECTIVE: To evaluate the diagnostic accuracy of USPSTF lung cancer screening eligibility criteria in a predominantly African American and low-income cohort. DESIGN, SETTING, AND PARTICIPANTS: The Southern Community Cohort Study prospectively enrolled adults visiting community health centers across 12 southern US states from March 25, 2002, through September 24, 2009, and followed up for cancer incidence through December 31, 2014. Participants included African American and white current and former smokers aged 40 through 79 years. Statistical analysis was performed from May 11, 2016, to December 6, 2018. EXPOSURES: Self-reported race, age, and smoking history. Cumulative exposure smoking histories encompassed most recent follow-up questionnaires. MAIN OUTCOMES AND MEASURES: Incident lung cancer cases assessed for eligibility for lung cancer screening using USPSTF criteria. RESULTS: Among 48 364 ever smokers, 32 463 (67%) were African American and 15 901 (33%) were white, with 1269 incident lung cancers identified. Among all 48 364 Southern Community Cohort Study participants, 5654 of 32 463 African American smokers (17%) were eligible for USPSTF screening compared with 4992 of 15 901 white smokers (31%) (P < .001). Among persons diagnosed with lung cancer, a significantly lower percentage of African American smokers (255 of 791; 32%) was eligible for screening compared with white smokers (270 of 478; 56%) (P < .001). The lower percentage of eligible lung cancer cases in African American smokers was primarily associated with fewer smoking pack-years among African American vs white smokers (median pack-years: 25.8 [interquartile range, 16.9-42.0] vs 48.0 [interquartile range, 30.2-70.5]; P < .001). Racial disparity was observed in the sensitivity and specificity of USPSTF guidelines between African American and white smokers for all ages. Lowering the smoking pack-year eligibility criteria to a minimum 20-pack-year history was associated with an increased percentage of screening eligibility of African American smokers and with equitable performance of sensitivity and specificity compared with white smokers across all ages (for a 55-year-old current African American smoker, sensitivity increased from 32.2% to 49.0% vs 56.5% for a 55-year-old white current smoker; specificity decreased from 83.0% to 71.6% vs 69.4%; P < .001). CONCLUSIONS AND RELEVANCE: Current USPSTF lung cancer screening guidelines may be too conservative for African American smokers. The findings suggest that race-specific adjustment of pack-year criteria in lung cancer screening guidelines would result in more equitable screening for African American smokers at high risk for lung cancer.

A Regression Framework for Causal Mediation Analysis with Applications to Behavioral Science.

We introduce and extend the classical regression framework for conducting mediation analysis from the fit of only one model. Using the essential mediation components (EMCs) allows us to estimate causal mediation effects and their analytical variance. This single-equation approach reduces computation time and permits the use of a rich suite of regression tools that are not easily implemented on a system of three equations. Additionally, we extend this framework to non-nested mediation systems, provide a joint measure of mediation for complex mediation hypotheses, propose new visualizations for mediation effects, and explain why estimates of the total effect may differ depending on the approach used. Using data from social science studies, we also provide extensive illustrations of the usefulness of this framework and its advantages over traditional approaches to mediation analysis. The example data are freely available for download online and we include the R code necessary to reproduce our results.

The GRE over the entire range of scores lacks predictive ability for PhD outcomes in the biomedical sciences.

The association between GRE scores and academic success in graduate programs is currently of national interest. GRE scores are often assumed to be predictive of student success in graduate school. However, we found no such association in admission data from Vanderbilt's Initiative for Maximizing Student Diversity (IMSD), which recruited historically underrepresented students for graduate study in the biomedical sciences at Vanderbilt University spanning a wide range of GRE scores. This study avoids the typical biases of most GRE investigations of performance where primarily high-achievers on the GRE were admitted. GRE scores, while collected at admission, were not used or consulted for admission decisions and comprise the full range of percentiles, from 1% to 91%. We report on the 32 students recruited to the Vanderbilt IMSD from 2007-2011, of which 28 completed the PhD to date. While the data set is not large, the predictive trends between GRE and long-term graduate outcomes (publications, first author publications, time to degree, predoctoral fellowship awards, and faculty evaluations) are remarkably null and there is sufficient precision to rule out even mild relationships between GRE and these outcomes. Career outcomes are encouraging; many students are in postdocs, and the rest are in regular stage-appropriate career environments for such a cohort, including tenure track faculty, biotech and entrepreneurship careers.

Protein structure aids predicting functional perturbation of missense variants in SCN5A and KCNQ1.

Rare variants in the cardiac potassium channel KV7.1 (KCNQ1) and sodium channel NaV1.5 (SCN5A) are implicated in genetic disorders of heart rhythm, including congenital long QT and Brugada syndromes (LQTS, BrS), but also occur in reference populations. We previously reported two sets of NaV1.5 (n = 356) and KV7.1 (n = 144) variants with in vitro characterized channel currents gathered from the literature. Here we investigated the ability to predict commonly reported NaV1.5 and KV7.1 variant functional perturbations by leveraging diverse features including variant classifiers PROVEAN, PolyPhen-2, and SIFT; evolutionary rate and BLAST position specific scoring matrices (PSSM); and structure-based features including "functional densities" which is a measure of the density of pathogenic variants near the residue of interest. Structure-based functional densities were the most significant features for predicting NaV1.5 peak current (adj. R2 = 0.27) and KV7.1 + KCNE1 half-maximal voltage of activation (adj. R2 = 0.29). Additionally, use of structure-based functional density values improves loss-of-function classification of SCN5A variants with an ROC-AUC of 0.78 compared with other predictive classifiers (AUC = 0.69; two-sided DeLong test p = .01). These results suggest structural data can inform predictions of the effect of uncharacterized SCN5A and KCNQ1 variants to provide a deeper understanding of their burden on carriers.

Volume at First Leak Is Associated With Sling Failure Among Women With Stress Urinary Incontinence.

INTRODUCTION: Stress urinary incontinence at a low bladder volume is a clinically observed phenomenon that is not well studied with regard to treatment outcomes. The primary aim of our study was to determine if the volume at first leak is associated with sling outcome. METHODS: This is a retrospective cohort study evaluating whether urodynamic stress urinary incontinence observed at low volumes is associated with sling failure using the Synthetic Derivative database. Sling failure was defined as (1) undergoing a subsequent surgery for stress incontinence (eg, urethral bulking agent, repeat sling) or (2) leakage that was subjectively worse or unchanged from baseline. Sling success was defined as subjective improvement in incontinence or being dry. Intrinsic sphincter deficiency was defined as maximum urethral closure pressure 20 cm H20 or less or abdominal leak point pressure less than 60 cm H20. RESULTS: Outcome data were available for 168 of 206 women who underwent a sling after urodynamic testing from 2006 to 2014. Of the 168 women, 80 were transobturator, 79 were retropubic, 8 lacked data regarding the approach to the midurethral sling, and 1 was an autologous pubovaginal sling. Similar failure rates were seen for transobturator (10%) and retropubic slings (7.6%). Preoperative urodynamic parameters, such as cystometric capacity and intrinsic sphincter deficiency, were similar among failed and successful slings. For every additional 50 mL in bladder volume at first leak (SUIvol), there was a 1.6 increased odds of having a successful sling (odds ratio, 1.576; 95% confidence interval, 1.014-2.450; P = 0.04). There was no statistically significant association between maximum urethral closure pressure, abdominal leak point pressure, body mass index, age, sling type, or whether a prior anti-incontinence procedure had been performed and sling success. CONCLUSIONS: Bladder volume at first leak is a strong predictor of sling failure.